Document de-registration

ABSTRACT

A document accessible over a network can be registered. A registered document, and the content contained therein, cannot be transmitted undetected over and off of the network. In one embodiment, the invention includes maintaining a plurality of stored signatures in a signature database, each signature being associated with one of a plurality of registered documents. In one embodiment, the invention further includes maintaining the signature database by de-registering documents by removing the signatures associated with de-registered documents. In one embodiment, the invention further includes maintaining the database by removing redundant and high detection rate signatures. In one embodiment, the invention also includes maintaining the signature database by removing signatures based on the source text used to generate the signature.

PRIORITY AND RELATED APPLICATIONS

This patent application is related to, incorporates by reference, andclaims the priority benefit of U.S. Provisional Application 60/528,631,entitled “DOCUMENT REGISTRATION”, filed Dec. 10, 2003.

FIELD OF THE INVENTION

The present invention relates to computer networks, and in particular,to registering documents in a computer network.

BACKGROUND

Computer networks and systems have become indispensable tools for modernbusiness. Modern enterprises use such networks for communications andfor storage. The information and data stored on the network of abusiness enterprise is often a highly valuable asset. Modern enterprisesuse numerous tools to keep outsiders, intruders, and unauthorizedpersonnel from accessing valuable information stored on the network.These tools include firewalls, intrusion detection systems, and packetsniffer devices. However, once an intruder has gained access tosensitive content, there is no network device that can prevent theelectronic transmission of the content from the network to outside thenetwork. Similarly, there is no network device that can analyse the dataleaving the network to monitor for policy violations, and make itpossible to track down information leeks. What is needed is acomprehensive system to capture, store, and analyse all datacommunicated using the enterprises network.

SUMMARY OF THE INVENTION

A document accessible over a network can be registered. A registereddocument, and the content contained therein, cannot be transmittedundetected over and off of the network. In one embodiment, the inventionincludes maintaining a plurality of stored signatures in a signaturedatabase, each signature being associated with one of a plurality ofregistered documents. In one embodiment, the invention further includesmaintaining the signature database by de-registering documents byremoving the signatures associated with de-registered documents. In oneembodiment, the invention further includes maintaining the database byremoving redundant and high detection rate signatures. In oneembodiment, the invention also includes maintaining the signaturedatabase by removing signatures based on the source text used togenerate the signature.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is illustrated by way of example, and not by wayof limitation, in the figures of the accompanying drawings in which likereference numerals refer to similar elements and in which:

FIG. 1 is a block diagram illustrating a computer network connected tothe Internet;

FIG. 2 is a block diagram illustrating one configuration of a capturesystem according to one embodiment of the present invention;

FIG. 3 is a block diagram illustrating the capture system according toone embodiment of the present invention;

FIG. 4 is a block diagram illustrating an object assembly moduleaccording to one embodiment of the present invention;

FIG. 5 is a block diagram illustrating an object store module accordingto one embodiment of the present invention;

FIG. 6 is a block diagram illustrating an example hardware architecturefor a capture system according to one embodiment of the presentinvention;

FIG. 7 is a block diagram illustrating a document registration systemaccording to one embodiment of the present invention;

FIG. 8 is a block diagram illustrating registration module according toone embodiment of the present invention;

FIG. 9 is a flow diagram illustrating object capture processingaccording to one embodiment of the present invention;

FIG. 10 is a flow diagram illustrating document de-registrationprocessing according to one embodiment of the present invention;

FIG. 11 is a flow diagram illustrating signature database maintenanceaccording to one embodiment of the present invention; and

FIG. 12 is a flow diagram illustrating signature database maintenanceaccording to another embodiment of the present invention.

DETAILED DESCRIPTION

Although the present system will be discussed with reference to variousillustrated examples, these examples should not be read to limit thebroader spirit and scope of the present invention. Some portions of thedetailed description that follows are presented in terms of algorithmsand symbolic representations of operations on data within a computermemory. These algorithmic descriptions and representations are the meansused by those skilled in the computer science arts to most effectivelyconvey the substance of their work to others skilled in the art. Analgorithm is here, and generally, conceived to be a self-consistentsequence of steps leading to a desired result. The steps are thoserequiring physical manipulations of physical quantities. Usually, thoughnot necessarily, these quantities take the form of electrical ormagnetic signals capable of being stored, transferred, combined,compared and otherwise manipulated.

It has proven convenient at times, principally for reasons of commonusage, to refer to these signals as bits, values, elements, symbols,characters, terms, numbers or the like. It should be borne in mind,however, that all of these and similar terms are to be associated withthe appropriate physical quantities and are merely convenient labelsapplied to these quantities. Unless specifically stated otherwise, itwill be appreciated that throughout the description of the presentinvention, use of terms such as “processing”, “computing”,“calculating”, “determining”, “displaying” or the like, refer to theaction and processes of a computer system, or similar electroniccomputing device, that manipulates and transforms data represented asphysical (electronic) quantities within the computer system's registersand memories into other data similarly represented as physicalquantities within the computer system memories or registers or othersuch information storage, transmission or display devices.

As indicated above, one embodiment of the present invention isinstantiated in computer software, that is, computer readableinstructions, which, when executed by one or more computerprocessors/systems, instruct the processors/systems to perform thedesignated actions. Such computer software may be resident in one ormore computer readable media, such as hard drives, CD-ROMs, DVD-ROMs,read-only memory, read-write memory and so on. Such software may bedistributed on one or more of these media, or may be made available fordownload across one or more computer networks (e.g., the Internet).Regardless of the format, the computer programming, rendering andprocessing techniques discussed herein are simply examples of the typesof programming, rendering and processing techniques that may be used toimplement aspects of the present invention. These examples should in noway limit the present invention, which is best understood with referenceto the claims that follow this description.

Networks

FIG. 1 illustrates a simple prior art configuration of a local areanetwork (LAN) 10 connected to the Internet 12. Connected to the LAN 102are various components, such as servers 14, clients 16, and switch 18.There are numerous other known networking components and computingdevices that can be connected to the LAN 10. The LAN 10 can beimplemented using various wireline or wireless technologies, such asEthernet and 802.11b. The LAN 10 may be much more complex than thesimplified diagram in FIG. 1, and may be connected to other LANs aswell.

In FIG. 1, the LAN 10 is connected to the Internet 12 via a router 20.This router 20, can be used to implement a firewall, which are widelyused to give users of the LAN 10 secure access to the Internet 12 aswell as to separate a company's public Web server (can be one of theservers 14) from its internal network, i.e., LAN 10. In one embodiment,any data leaving the LAN 10 towards the Internet 12 must pass throughthe router 12. However, there the router 20 merely forwards packets tothe Internet 12. The router 20 cannot capture, analyse, and searchablystore the content contained in the forwarded packets.

One embodiment of the present invention is now illustrated withreference to FIG. 2. FIG. 2 shows the same simplified configuration ofconnecting the LAN 10 to the Internet 12 via the router 20. However, inFIG. 2, the router 20 is also connected to a capture system 22. In oneembodiment, the router 12 splits the outgoing data stream, and forwardsone copy to the Internet 12 and the other copy to the capture system 22.

There are various other possible configurations. For example, the router12 can also forward a copy of all incoming data to the capture system 22as well. Furthermore, the capture system 22 can be configuredsequentially in front of, or behind the router 20, however this makesthe capture system 22 a critical component in connecting to the Internet12. In systems where a router 12 is not used at all, the capture systemcan be interposed directly between the LAN 10 and the Internet 12. Inone embodiment, the capture system 22 has a user interface accessiblefrom a LAN-attached device, such as a client 16.

In one embodiment, the capture system 22 intercepts all data leaving thenetwork. In other embodiments, the capture system can also intercept alldata being communicated inside the network 10. In one embodiment, thecapture system 22 reconstructs the documents leaving the network 10, andstores them in a searchable fashion. The capture system 22 can then beused to search and sort through all documents that have left the network10. There are many reasons such documents may be of interest, includingnetwork security reasons, intellectual property concerns, corporategovernance regulations, and other corporate policy concerns.

Capture System

One embodiment of the present invention is now described with referenceto FIG. 3. FIG. 3 shows one embodiment of the capture system 22 in moredetail. The capture system 22 is also sometimes referred to as a contentanalyzer, content or data analysis system, and other similar names. Inone embodiment, the capture system 22 includes a network interfacemodule 24 to receive the data from the network 10 or the router 20. Inone embodiment, the network interface module 24 is implemented using oneor more network interface cards (NIC), e.g., Ethernet cards. In oneembodiment, the router 20 delivers all data leaving the network to thenetwork interface module 24.

The captured raw data is then passed to a packet capture module 26. Inone embodiment, the packet capture module 26 extracts data packets fromthe data stream received from the network interface module 24. In oneembodiment, the packet capture module 26 reconstructs Ethernet packetsfrom multiple sources to multiple destinations for the raw data stream.

In one embodiment, the packets are then provided the object assemblymodule 28. The object assembly module 28 reconstructs the objects beingtransmitted by the packets. For example, when a document is transmitted,e.g. as an email attachment, it is broken down into packets according tovarious data transfer protocols such as Transmission ControlProtocol/Internet Protocol (TCP/IP) and Ethernet. The object assemblymodule 28 can reconstruct the document from the captured packets.

One embodiment of the object assembly module 28 is now described in moredetail with reference to FIG. 4. When packets first enter the objectassembly module, they are first provided to a reassembler 36. In oneembodiment, the reassembler 36 groups—assembles—the packets into uniqueflows. For example, a flow can be defined as packets with identicalSource IP and Destination IP addresses as well as identical TCP Sourceand Destination Ports. That is, the reassembler 36 can organize a packetstream by sender and recipient.

In one embodiment, the reassembler 36 begins a new flow upon theobservation of a starting packet defined by the data transfer protocol.For a TCP/IP embodiment, the starting packet is generally referred to asthe “SYN” packet. The flow can terminate upon observation of a finishingpacket, e.g., a “Reset” or “FIN” packet in TCP/IP. If now finishingpacket is observed by the reassembler 36 within some time constraint, itcan terminate the flow via a timeout mechanism. In an embodiment usingthe TPC protocol, a TCP flow contains an ordered sequence of packetsthat can be assembled into a contiguous data stream by the ressembler36. Thus, in one embodiment, a flow is an ordered data stream of asingle communication between a source and a destination.

The flown assembled by the reassember 36 can then be provided to aprotocol demultiplexer (demux) 38. In one embodiment, the protocol demux38 sorts assembled flows using the TCP Ports. This can includeperforming a speculative classification of the flow contents based onthe association of well-known port numbers with specified protocols. Forexample, Web Hyper Text Transfer Protocol (HTTP) packets—i.e., Webtraffic—are typically associated with port 80, File Transfer Protocol(FTP) packets with port 20, Kerberos authentication packets with port88, and so on. Thus in one embodiment, the protocol demux 38 separatesall the different protocols in one flow.

In one embodiment, a protocol classifier 40 also sorts the flows inaddition to the protocol demux 38. In one embodiment, the protocolclassifier 40—operating either in parallel or in sequence with theprotocol demux 38—applies signature filters to the flows to attempt toidentify the protocol based solely on the transported data. Furthermore,the protocol demux 38 can make a classification decision based on portnumber which is subsequently overridden by protocol classifier 40. Forexample, if an individual or program attempted to masquerade an illicitcommunication (such as file sharing) using an apparently benign portsuch as port 80 (commonly used for HTTP Web browsing), the protocolclassifier 40 would use protocol signatures, i.e., the characteristicdata sequences of defined protocols, to verify the speculativeclassification performed by protocol demux 38.

In one embodiment, the object assembly module 28 outputs each floworganized by protocol, which represent the underlying objects. Referringagain to FIG. 3, these objects can then be handed over to the objectclassification module 30 (sometimes also referred to as the “contentclassifier”) for classification based on content. A classified flow maystill contain multiple content objects depending on the protocol used.For example, protocols such as HTTP (Internet Web Surfing) may containover 100 objects of any number of content types in a single flow. Todeconstruct the flow, each object contained in the flow is individuallyextracted, and decoded, if necessary, by the object classificationmodule 30.

The object classification module 30 uses the inherent properties andsignatures of various documents to determine the content type of eachobject. For example, a Word document has a signature that is distinctfrom a PowerPoint document, or an Email document. The objectclassification module 30 can extract out each individual object and sortthem out by such content types. Such classification renders the presentinvention immune from cases where a malicious user has altered a fileextension or other property in an attempt to avoid detection of illicitactivity.

In one embodiment, the object classification module 30 determineswhether each object should be stored or discarded. In one embodiment,this determination is based on a various capture rules. For example, acapture rule can indicate that Web Traffic should be discarded. Anothercapture rule can indicate that all PowerPoint documents should bestored, except for ones originating from the CEO's IP address. Suchcapture rules can be implemented as regular expressions, or by othersimilar means.

In one embodiment, the capture rules are authored by users of thecapture system 22. The capture system 22 is made accessible to anynetwork-connected machine through the network interface module 24 anduser interface 34. In one embodiment, the user interface 34 is agraphical user interface providing the user with friendly access to thevarious features of the capture system 22. For example, the userinterface 34 can provide a capture rule authoring tool that allows usersto write and implement any capture rule desired, which are then appliedby the object classification module 30 when determining whether eachobject should be stored. The user interface 34 can also providepre-configured capture rules that the user can select from along with anexplanation of the operation of such standard included capture rules. Inone embodiment, the default capture rule implemented by the objectclassification module 30 captures all objects leaving the network 10.

If the capture of an object is mandated by the capture rules, the objectclassification module 30 can also determine where in the object storemodule 32 the captured object should be stored. With reference to FIG.5, in one embodiment, the objects are stored in a content store 44memory block. Within the content store 44 are files 46 divided up bycontent type. Thus, for example, if the object classification moduledetermines that an object is a Word document that should be stored, itcan store it in the file 46 reserved for Word documents. In oneembodiment, the object store module 32 is integrally included in thecapture system 22. In other embodiments, the object store module can beexternal—entirely or in part—using, for example, some network storagetechnique such as network attached storage (NAS) and storage areanetwork (SAN).

In one embodiment, the content store is a canonical storage location,simply a place to deposit the captured objects. The indexing of theobjects stored in the content store 44 is accomplished using a tagdatabase 42. In one embodiment, the tag database 42 is a database datastructure in which each record is a “tag” that indexes an object in thecontent store 44, and contains relevant information about the storedobject. An example of a tag record in the tag database 42 that indexesan object stored in the content store 44 is set forth in Table 1:

TABLE 1 Field Name Definition MAC Address Ethernet controller MACaddress unique to each capture system Source IP Source Ethernet IPAddress of object Destination IP Destination Ethernet IP Address ofobject Source Port Source TCP/IP Port number of object Destination PortDestination TCP/IP Port number of the object Protocol IP Protocol thatcarried the object Instance Canonical count identifying object within aprotocol capable of carrying multiple data within a single TCP/IPconnection Content Content type of the object Encoding Encoding used bythe protocol carrying object Size Size of object Timestamp Time that theobject was captured Owner User requesting the capture of object (ruleauthor) Configuration Capture rule directing the capture of objectSignature Hash signature of object Tag Signature Hash signature of allpreceding tag fields

There are various other possible tag fields, and some embodiments canomit numerous tag fields listed in Table 1. In other embodiments, thetag database 42 need not be implemented as a database; other datastructures can be used. The mapping of tags to objects can, in oneembodiment, be obtained by using unique combinations of tag fields toconstruct an object's name. For example, one such possible combinationis an ordered list of the Source IP, Destination IP, Source Port,Destination Port, Instance and Timestamp. Many other such combinationsincluding both shorter and longer names are possible. In anotherembodiment, the tag can contain a pointer to the storage location wherethe indexed object is stored.

Referring again to FIG. 3, in one embodiment, the objects and tagsstored in the object store module 32 can be interactively queried by auser via the user interface 34. In one embodiment the user interface caninteract with a web server (not shown) to provide the user withWeb-based access to the capture system 22. The objects in the contentstore module 32 can thus be searched for specific textual or graphicalcontent using exact matches, patterns, keywords, and various otheradvanced attributes.

For example, the user interface 34 can provide a query-authoring tool(not shown) to enable users to create complex searches of the objectstore module 32. These search queries can be provided to a data miningengine (not shown) that parses the queries, scans the tag database 42,and retrieves the found object from the content store 44. Then, theseobjects that matched the specific search criteria in the user-authoredquery can be counted and displayed to the user by the user interface 34.

Searches can also be scheduled to occur at specific times or at regularintervals, that is, the user interface 34 can provide access to ascheduler (not shown) that can periodically execute specific queries.Reports containing the results of these searches can be made availableto the user at a later time, mailed to the administrator electronically,or used to generate an alarm in the form of an e-mail message, page,syslog or other notification format.

In several embodiments, the capture system 22 has been described aboveas a stand-alone device. However, the capture system of the presentinvention can be implemented on any appliance capable of capturing andanalysing data from a network. For example, the capture system 22described above could be implemented on one or more of the servers 14 orclients 16 shown in FIG. 1. The capture system 22 can interface with thenetwork 10 in any number of ways, including wirelessly.

In one embodiment, the capture system 22 is an appliance constructedusing commonly available computing equipment and storage systems capableof supporting the software requirements. In one embodiment, illustratedby FIG. 6, the hardware consists of a capture entity 46, a processingcomplex 48 made up of one or more processors, a memory complex 50 madeup of one or more memory elements such as RAM and ROM, and storagecomplex 52, such as a set of one or more hard drives or other digital oranalog storage means. In another embodiment, the storage complex 52 isexternal to the capture system 22, as explained above. In oneembodiment, the memory complex stored software consisting of anoperating system for the capture system device 22, a capture program,and classification program, a database, a filestore, an analysis engineand a graphical user interface.

Document Registration

The capture system 22 described above can also be used to implement adocument registration scheme. In one embodiment, the a user can registera document with the system 22, which can then alert the user if all orpart of the content in the registered document is leaving the network.Thus, one embodiment of the present invention aims to preventun-authorized documents of various formats (e.g., Microsoft Word, Excel,PowerPoint, source code of any kind, text) from leaving an enterprise.There are great benefits to any enterprise that can keep itsintellectual property, or other critical, confidential, or otherwiseprivate and proprietary content from being mishandled.

In one embodiment of the present invention, sensitive documents areregistered with the capture system 22, although data registration can beimplemented using a separate device in other embodiments. One embodimentof implementing registration capability in the capture system 22 is nowdescribed with reference to FIG. 7. For descriptive purposes, thecapture system 22 is renamed the capture/registration system 22 in FIG.7, and is also sometimes referred to as the registration system 22 inthe description herein. The capture/registration system 22 hascomponents similar or identical to the capture system 22 shown in FIG.3, including the network interface module 24, the object store module32, the user interface 34, and the packet capture 26, object assembly28, and object classification 30 modules, which are grouped together asobject capture modules 31 in FIG. 7.

In one embodiment, the capture/registration system 22 also includes aregistration module 54 interacting with a signature database 56 tofacilitate a registration scheme. In one embodiment, the user canregister a document via the user interface 34. There are numerous waysto register documents. For example, a document can be electricallymailed (e-mailed), or uploaded to the registration system 22. Theregistration system 22 can also periodically scan a file server(registration server) for documents to be registered. The registrationprocess can be integrated with the enterprise's document managementsystems. Document registration can also be automated and transparentbased on registration rules, such as “register all documents,” or“register all documents by specific author or IP address,” and so on.

After being received, in one embodiment, a document to be registered ispassed to the registration module 54. The registration module 54calculates a signature of the document, or a set of signatures. The setof signatures associated with the document can be calculated in variousways. For example, the signatures can be made up of hashes over variousportions of the document, such as selected or all pages, paragraphs,tables and sentences. Other possible signatures include, but are notlimited to, hashes over embedded content, indices, headers or footers,formatting information or font utilization. The signatures can alsoinclude computations and meta-data other than hash digests, such as WordRelative Frequency Methods (RFM)—Statistical, Karp-RabinGreedy-String-Tiling-Transposition, vector space models, anddiagrammatic structure analysis.

The set of signatures is then stored in the signature database 56. Thesignature database 56 need not be implemented as a database; thesignatures can be maintained using any appropriate data structure. Inone embodiment, the signature database 56 is part of the storage complex52 in FIG. 6.

In one embodiment, the registered document is also stored as an objectin the object store module 32. In one embodiment, the document is onlystored in the content store 44 with no associated tag, since many tagfields do not apply to registered documents. In one embodiment, one fileof files 46 is a “Registered Documents” file.

In one embodiment, the document received from the user is nowregistered. As set forth above, in one embodiment, the object capturemodules 31 continue to extract objects leaving the network, and storevarious objects based on capture rules. In one embodiment, all extractedobjects—whether subject to a capture rule or not—are also passed to theregistration module for a determination whether each object is, orincludes part of, a registered document.

In one embodiment, the registration module 54 calculates the set ofsignatures of an object received from the object capture modules 31 inthe same manner as of a document received from the user interface 34 tobe registered. This set of signatures is then compared against allsignatures in the signature database 56. In other embodiment, parts ofthe signature database can be excluded from this search to save time.

In one embodiment, an unauthorized transmission is detected if any oneor more signatures in the set of signatures of an extracted objectmatches one or more signature in the signature database 56 associatedwith a registered document. Other detection tolerances can be configuredfor different embodiment, e.g., at least two signatures must match.Also, special rules can be implemented that make the transmissionauthorized, e.g., if the source address is authorized to transmit anydocuments off the network.

One embodiment of the registration module 54 is now described withreference to FIG. 8. As discussed above, a document to be registered 68arrives via the user interface 34. The registration engine 58 generatessignatures 60 for the document 68 and forwards the document 68 to thecontent store 44 and the signatures 60 to the signature database 54. Thesignatures 60 are associated with the document, e.g., by including apointer to the document 68, or to some attribute from which the document68 can be identified.

A captured object 70 arrives via the object capture modules 31. Theregistration engine calculates the signatures 62 of the captured object,and forwards them to the search engine 64. The search engine 64 queriesthe signature database 54 to compare the signatures 62 to the signaturesstored in the signature database 54. Assuming for the purposes ofillustration, that the captured object 70 is a Word document thatcontains a pasted paragraph from registered PowerPoint document 68, atleast one signature of signatures 62 will match a signature ofsignatures 60. Such an event can be referred to as the detection of anunauthorized transfer, a registered content transfer, or other similarlydescriptive terms.

In one embodiment, when a registered content transfer is detected, thetransmission can be halted with or without warning to the sender. In oneembodiment, in the event of a detected registered content transfer, thesearch engine 64 activates the notification module 66, which sends analert 72 to the user via the user interface 34. In one embodiment, thenotification module 66 sends different alerts—including different useroptions—based on the user preference associated with the registration,and the capabilities of the registration system 22.

In one embodiment, the alert 72 can simply indicate that the registeredcontent, i.e., the captured object 70, has been transferred off thenetwork. In addition, the alert 72 can provide information regarding thetransfer, such as source IP, destination IP, any other informationcontained in the tag of the captured object, or some other derivedinformation, such as the name of the person who transferred the documentoff the network. The alert 72 can be provided to one or more users viae-mail, instant message (IM), page, or any other notification method. Inone embodiment, the alert 72 is only sent to the entity or user whorequested registration of the document 68.

In another embodiment, the delivery of the captured object 70 ishalted—the transfer is not completed—unless the user who registered thedocument 68 consents. In such an embodiment, the alert 72 can containall information described above, and in addition, contain a selectionmechanism, such as one or two buttons—to allow the user to indicatewhether the transfer of the captured object 70 may be completed. If theuser elects to allow the transfer, for example because he is aware thatsomeone is emailing a part of a registered document 68 (e.g., a bossasking his secretary to send an email), the transfer is executed and theobject 70 is allowed to leave the network.

If the user disallows the transfer, the captured object 70 is notallowed off the network, and delivery is permanently halted. In oneembodiment, halting delivery can be accomplished by implementing anintercept technique by having the registration system 22 proxy theconnection between the network and the outside. In other embodiments,delivery can be halted using a black hole technique—discarding thepackets without notice if the transfer is disallowed—or a poisontechnique—inserting additional packets onto the network to cause thesender's connection to fail.

FIG. 9 provides a flow chart to further illustrate objectcapture/intercept processing according to one embodiment of the presentinvention. All blocks of FIG. 9 have already been discussed herein. Theexample object capture processing shown in FIG. 9 assumes that variousdocuments have already been registered with the registration system 22.The process shown in FIG. 9 can be repeated for all objects captured bythe system 22.

Document De-Registration and Signature Database Maintenance

In one embodiment, documents can also be de-registered when theircontent is no longer considered worthy of registration by a user.Various other signature database 54 maintenance processes can beimplemented. Referring again to FIG. 8, in one embodiment, theregistration engine 58 is also used to de-register documents.De-registration can be performed similarly to registration andinterception, with the end result being the removal, as opposed to theaddition of, the signatures generated.

Thus, in one embodiment, when the registration engine 58 receives adocument to be de-registered it generates a set of signatures associatedwith the document, and provides these signatures to the search engine64. The search engine 64 identifies the signatures in the signaturedatabase 54 that match any of the signatures associated with thedocument to be de-registered. Then, these matching signatures areremoved from the signature database, resulting in the de-registration ofthe document. A simplified flowchart illustrating such an embodiment isprovided in FIG. 10. The next time the document is intercepted, none ofits signatures will match any in the database.

Document de-registration can also be performed, in one embodiment, usingthe name of the document the user wishes to de-register. As explainedabove, the name of a document or object can be constructed to be uniquein a number of ways, such as combining several tag fields. Since thesignatures in the signature database 54 are associated with documents inthe object store module 32, the set of signatures associated with auniquely identified document are readily ascertainable. Thus, in oneembodiment, the user can provide a name of a document to bede-registered via the user interface 34, in response to which allsignatures associated with that document are removed from the signaturedatabase 54.

Document de-registration can be automated. In one embodiment, theregistration rules also include temporal registration parameters. Thus,certain documents only stay registered for some period of time afterwhich they become automatically de-registered by any of thede-registration procedures described herein. Other such de-registrationrules can be authored and implemented by the system.

In one embodiment, aspects of signature database 54 maintenance are alsoautomated. One embodiment of automated database maintenance is nowdescribed with reference to FIG. 11. In block 1110, the signaturedetection rate is observed for each signature. In one embodiment, thesignature detection rate signifies how often a signature is detected incaptured objects. It can be implemented as a canonical count, as somenumber of occurrences during specified periods of time, a percentage ofcaptured objects that triggers the signature, or some other rate ofoccurrence indicator.

In block 1120, a signature—or set of signatures—with a high rate ofdetection is identified. This can be done by observing that thedetection rate of some signatures exceeds a configurable rate threshold.For example, it is observed that a signature is found in 50 percent ofall captured objects, and the rate threshold is 10 percent. Thissignature would be identified as a high-rate signature. Finally, inblock 1130, the identified high-rate signatures are removed from thesignature database.

In one embodiment, database maintenance is further automated by periodicexamination of the source data resulting in the signatures. For example,a signature may result from source data contained in the registereddocument that is irrelevant text (e.g., copyright notice), unparsabletext, or text contained on a signature blacklist identifying text thatshould not generate signatures.

For example, the signature blacklist may contain specific words,phrases, or set of words that are known to result in unusable signatures(e.g., high-rate signatures). Similarly, a whitelist is maintained, inone embodiment, containing text that should be used for signatures. Forexample, if a signature resulted from source data contained in thewhitelist, it may not be removed for other reasons, such as being ahigh-rate signature.

In one embodiment, maintaining the signature database 54 includesperiodically eliminating redundant signatures from the database 54.Redundant signatures are identical, but are associated with differentregistered documents. For example, if a registration rule registers allsource code files, then the comment header will result in an identicalsignature (based on the identical comment header) being associated withall these registered documents.

One problem that can arise from such a situation, is that when a sourcecode file is intercepted, one signature (based on the comment header)will match a great number of signatures associated with differentregistered documents. In one embodiment, this is solved by removingredundant signatures from the database 54. In one embodiment, redundantsignatures are defined as a set of identical signatures common to morethan a threshold number of registered documents. For example, if morethan five registered documents have an associated identical signature,then all of these redundant signatures are removed from the database 54.

In one embodiment, signatures stored in the signature database 54 areaccessible via the user interface 34. Thus, a user can individuallyselect and remove signatures from the database 34. For example, if auser observes that a signature is generating too many false-positives(noticing irrelevant registered content), then the user can remove thatsignature from the database 34. An illustration of removing redundantsignatures is provided in the form of a simplified flowchart in FIG. 12.

Thus, a capture system and a document/content registration system havebeen described. In the forgoing description, various specific valueswere given names, such as “objects,” and various specific modules, suchas the “registration module” and “signature database” have beendescribed. However, these names are merely to describe and illustratevarious aspects of the present invention, and in no way limit the scopeof the present invention. Furthermore, various modules, such as thesearch engine 64 and the notification module 66 in FIG. 8, can beimplemented as software or hardware modules, or without dividing theirfunctionalities into modules at all. The present invention is notlimited to any modular architecture either in software or in hardware,whether described above or not.

1-54. (canceled)
 55. At least one machine accessible storage mediumhaving instructions stored thereon, the instructions when executed on amachine, cause the machine to: monitor security of a plurality ofregistered digital documents in a system, the monitoring includingdetermining whether signatures associated with the registered digitaldocuments are included in data propagating in network traffic of thesystem; detect a particular signature of a particular document in theplurality of registered digital documents from the data propagating inthe network; determine, based at least in part on the detecting, thatdetection of the particular signature exceeds a threshold detection ratefor registered digital documents in the system; and remove theparticular signature from a signature database including the signaturesof the plurality of registered digital documents.
 56. The storage mediumof claim 55, wherein data in the network traffic detected to include asignature of a registered document is to be intercepted and data in thenetwork traffic detected to not include a signature of a registereddocument is to be allowed to propagate unintercepted to its intendeddestination.
 57. The storage medium of claim 55, wherein detecting theparticular signature includes receiving the particular document,calculating the particular signature associated with the particulardocument, and determining whether the particular signature is includedin the signature database.
 58. The storage medium of claim 55, whereinremoving the particular signature from the signature databasede-registers the particular document.
 59. The storage medium of claim58, wherein another document in the plurality of registered digitaldocuments is to be deregistered based on an expiration of a timeinterval associated with an initial registration of the other document.60. The storage medium of claim 58, wherein de-registering theparticular document includes removing all of a plurality of signaturesassociated with the particular document from the signature database. 61.The storage medium of claim 55, wherein the threshold detection ratecomprises a threshold percentage of all data objects detected in thenetwork traffic.
 62. The storage medium of claim 55, wherein theinstructions, when executed by on a machine, are further to determinethat the particular signature is redundant with signatures associatedwith at least one other document in the plurality of registered digitaldocuments.
 63. The storage medium of claim 62, further comprisingremoving the particular signature from records of the signature databaseassociated with each of the registered digital documents associated withthe particular signature.
 64. The storage medium of claim 62, whereindetermining that the particular signature is a redundant signatureincludes identifying that a signature of the other document issubstantially identical to the particular signature.
 65. The storagemedium of claim 55, wherein the threshold detection rate comprises athreshold number of detections of signatures substantially identical tothe particular signature.
 66. The storage medium of claim 55, whereinthe threshold detection rate of the particular signature includes anumber of times the particular signature is detected from data objectsdetected in the network traffic during a time window.
 67. A methodcomprising: monitoring security of a plurality of registered digitaldocuments in a system, the monitoring including determining whethersignatures associated with the registered digital documents are includedin data propagating in network traffic of the system; detecting aparticular signature of a particular document in the plurality ofregistered digital documents from the data propagating in the network;determining, based at least in part on the detecting, that detection ofthe particular signature exceeds a threshold detection rate forregistered digital documents in the system; and removing the particularsignature from a signature database including the signatures of theplurality of registered digital documents.
 68. The method of claim 67,further comprising automatically de-registering the particular documentbased on determining that detection of the particular signature exceedsthe threshold detection rate.
 69. The method of claim 67, wherein thethreshold detection rate comprises a threshold percentage of all dataobjects detected in the network traffic.
 70. The method of claim 67,wherein the threshold detection rate includes a rate of signatureappearance per day.
 71. The method of claim 67, further comprisingintercepting registered documents included in the data propagating inthe network based on detection of a corresponding signature for theregistered documents.
 72. The method of claim 71, wherein data objectsthat do not have a corresponding signature stored in the signaturedatabase are allowed to propagate unintercepted to their respectiveintended destinations.
 73. A system comprising: at least one processordevice; at least one memory element; and a registration module to:monitor security of a plurality of registered digital documents in asystem, the monitoring including determining whether signaturesassociated with the registered digital documents are included in datapropagating in network traffic of the system; detect a particularsignature of a particular document in the plurality of registereddigital documents from the data propagating in the network; determine,based at least in part on the detecting, that detection of theparticular signature exceeds a threshold detection rate for registereddigital documents in the system; and remove the particular signaturefrom a signature database including the signatures of the plurality ofregistered digital documents.
 74. The system of claim 73, wherein theregistration module is further to designate an additional digitaldocument as a registered digital document, wherein designating theadditional digital document as a registered digital document comprisesgenerating one or more signatures for the additional digital documentand storing the signatures for the additional digital document in thesignature database.
 75. The system of claim 73, further comprising thesignature database.
 76. The system of claim 73, wherein the registrationmodule is further to de-register documents in the plurality ofregistered digital document based on determining that a time intervalassociated with initial registration of the registered documents hasexpired.